W3. Matrix Product Geometry

Author

Salman Ahmadi-Asl

Published

September 18, 2025

1. Theory

1.1 Introduction to Matrices

A matrix is a rectangular grid of numbers or expressions arranged in rows and columns. It serves as a powerful tool for organizing data and representing systems of linear equations or transformations. A matrix is denoted by a capital letter (e.g., ), and its individual entries are called elements.

The dimensions of a matrix are expressed as m x n, where m is the number of rows and n is the number of columns. An element at the intersection of the i-th row and j-th column is denoted by .

For example, a 2x3 matrix A is written as:

1.2 Types of Matrices

Matrices with specific structures and properties are given special names:

Square Matrix: A matrix where the number of rows equals the number of columns (m = n). These are the only matrices that have determinants and inverses.
Identity Matrix (): A square matrix with ones on the main diagonal (top-left to bottom-right) and zeros everywhere else. It is the matrix equivalent of the number 1; multiplying a matrix by the identity matrix leaves it unchanged ().
Zero Matrix: A matrix where every element is zero.
Diagonal Matrix: A square matrix where all elements not on the main diagonal are zero.
Symmetric Matrix: A square matrix that is unchanged by transposition (). This means that for any i and j, the element is equal to .
Triangular Matrix: A square matrix where all elements are zero either above or below the main diagonal. If the elements below the diagonal are zero, it is an upper triangular matrix. If the elements above are zero, it is a lower triangular matrix.
Hermitian Matrix: An extension of symmetric matrices for complex numbers. A matrix is Hermitian if it is equal to its own conjugate transpose (taking the transpose and then the complex conjugate of each element).

1.3 Basic Matrix Operations

Addition and Subtraction: These operations are only defined for matrices of the same dimensions. The operation is performed element-wise, meaning the corresponding elements of the matrices are added or subtracted. For , the elements are .
Scalar Multiplication: To multiply a matrix by a scalar (a single number), you multiply every element in the matrix by that scalar. For , the elements are .
Transpose (): The transpose of a matrix is found by interchanging its rows and columns. The first row becomes the first column, the second row becomes the second column, and so on. If A is an m x n matrix, is an n x m matrix. A key property is that the transpose of a product reverses the order: .
Trace (): The trace is defined only for a square matrix and is the sum of the elements on its main diagonal. A crucial property is its cyclic invariance under multiplication: .

1.4 Matrix Multiplication

Matrix multiplication is a more complex operation. For the product to be defined, the number of columns in matrix A must equal the number of rows in matrix B. If A is an m x n matrix and B is an n x p matrix, the resulting product C will be an m x p matrix.

The element in the resulting matrix is the dot product of the i-th row of A and the j-th column of B. There are two key ways to conceptualize matrix-vector multiplication ():

Dot Product View: Each element of the resulting vector is the dot product of a corresponding row of matrix A with the vector x.
Linear Combination View: The resulting vector is a weighted sum (a linear combination) of the columns of matrix A, where the weights are the elements of vector x.

A fundamental property of matrix multiplication is that it is not commutative: in general, . The order matters greatly. Think of it like real-world actions: putting on socks then shoes is very different from putting on shoes then socks. However, matrix multiplication is associative () and distributive ().

1.5 The Determinant of a Matrix

The determinant, denoted or , is a unique scalar value that can only be calculated from a square matrix. It provides crucial information:

Invertibility: A matrix has an inverse if and only if its determinant is non-zero. A matrix with a zero determinant is called a singular matrix.
Geometric Meaning: The determinant represents the scaling factor of the linear transformation described by the matrix. For a 2x2 matrix, its absolute value tells you how much area is scaled. For a 3x3 matrix, it’s the scaling factor for volume. A negative determinant indicates that the transformation also involves a reflection (an “inversion” of orientation).

Key properties of determinants include:

For an n x n matrix,

1.6 The Matrix Inverse

The inverse of a square matrix A, written as , is the matrix that “undoes” the operation of A. When multiplied together, they yield the identity matrix: . This is analogous to how a number multiplied by its reciprocal equals 1 (e.g., ). An inverse exists only if the matrix is non-singular ().

The adjugate method is a common way to find the inverse of a 3x3 matrix:

Matrix of Minors (): For each element , its minor is the determinant of the 2x2 sub-matrix that remains after deleting the i-th row and j-th column.
Matrix of Cofactors (): The cofactor is the minor with a sign applied, determined by a “checkerboard” pattern. The formula is .
Adjugate Matrix (): This is the transpose of the cofactor matrix, = .
Calculate Inverse: The inverse is found by multiplying the adjugate matrix by the reciprocal of the determinant: .

1.7 The Vector Cross Product

The cross product is an operation between two vectors in 3D space, written as a x b. The result is a new vector that is geometrically perpendicular (orthogonal) to both of the original vectors.

The direction of the resulting vector is given by the right-hand rule: point the fingers of your right hand in the direction of vector a, then curl them toward vector b. Your thumb will point in the direction of a x b. Key properties of the cross product:

Anti-commutative: Reversing the order flips the direction of the resulting vector: a x b = - (b x a).
Parallel Vectors: The cross product of two parallel vectors is the zero vector (0).
Magnitude: The magnitude, , equals the area of the parallelogram formed by vectors a and b. The area of the triangle with these sides is half this value.

1.8 Triple Products

Triple products combine dot and cross products for three vectors.

Scalar Triple Product: This product, , results in a scalar. Its absolute value represents the volume of the parallelepiped with sides a, b, and c. If the result is 0, the vectors are coplanar (they all lie in the same plane), meaning the parallelepiped is flat and has no volume. The operation is cyclically symmetric: .
Vector Triple Product: This product, , results in a vector. The resulting vector lies in the same plane as b and c. It can be simplified with the “BAC-CAB” identity: .

Another key identity is the Jacobi identity, which shows a cyclic relationship between triple products:

2. Definitions

Matrix: A rectangular array of numbers arranged in rows and columns.
Scalar: A single numerical quantity that scales a vector or matrix.
Square Matrix: A matrix with an equal number of rows and columns.
Identity Matrix (): A square matrix with ones on the main diagonal and zeros elsewhere, which acts as the multiplicative identity.
Determinant (): A scalar value computed from a square matrix that determines its invertibility and geometric scaling properties.
Singular Matrix: A square matrix whose determinant is zero, meaning it has no inverse.
Matrix Inverse (): For a non-singular square matrix A, it is the unique matrix that yields the identity matrix when multiplied with A.
Transpose (): A matrix formed by interchanging the rows and columns of the original matrix.
Trace (): The sum of the elements on the main diagonal of a square matrix.
Adjugate Matrix (): The transpose of the matrix of cofactors.
Coplanar Vectors: A set of vectors that all lie within the same two-dimensional plane.
Vector Cross Product (): A binary operation on two vectors in 3D space, resulting in a vector perpendicular to both.
Scalar Triple Product (): An operation whose absolute value is the volume of the parallelepiped defined by three vectors.
Vector Triple Product (): An operation involving three vectors that results in a new vector coplanar with the second and third vectors.